Entity linking system

ABSTRACT

A linking system that assists in managing the linking of entities contained in messages between two business applications is disclosed. The linking system also assists in updating entities between two business applications when an entity is updated in one application. The linking system takes each entity and places identifying information for the entity in a record in a first table. The system then creates in a record in a second table indicating that the two records in the first table are linked. The linking system also can remove links or records from the tables, and can return to a user a list of records that are linked together.

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority from U.S. Ser. No. 10/402379,which was filed Mar. 28, 2003, and was converted to ProvisionalApplication Serial No. 60/___,___ with a petition filed May __, 2003.

BACKGROUND OF THE INVENTION

[0002] The present invention relates to the integration of businessapplications in an integrated business solutions computing environment.More specifically, the present invention relates to maintaining theorder of messages and documents when processed or transformed by anenterprise server.

[0003] The current business environment is very different from what itwas just a few years ago. Today's organizations embrace the globalmarketplace, and this dictates a need to be able to efficiently operateat all times. Customers are now more sophisticated which translates intoan accelerated pace of business and decision-making processes. Further,business relationships have become highly dynamic, and customers expectbusinesses to adapt quickly.

[0004] Technical and operational challenges abound as well. There is aneed to support multiple applications on a variety of platforms, and tointegrate with companies using the Internet, extranets, business tobusiness (B2B) exchanges, and other resources. Also, to effectivelycompete in today's market, there is a need to build new solutions on“Internet time,” utilizing open Internet standards and technology toassure maximum interoperability.

[0005] Businesses have typically used a variety of mechanisms to controland analyze business operations such as accounting, payroll, humanresources, employee tracking, customer relations tracking, etc. Toolswhich provide these functions are often implemented using computersoftware. For example, a software package may manage businessaccounting, another software package might be responsible for receivingnew orders, yet another software package will track warehouse inventoryand still another package may handle order fulfillment and shipment. Inanother example, a business software package operated by one businesswill need to exchange data with a software package operated by anotherbusiness to allow a business-to-business transaction to occur.

[0006] When business tools are implemented in software, it is notunusual for proprietary software packages to be responsible for eachindividual business task. However, this implementation is cumbersome andrequires the same data to be entered in differing formats among thevarious business applications. In order to improve efficiency,integration applications have been developed which are used to integratevarious elements of one business application with elements of anotherbusiness application.

[0007] For example, if a software package, which is used to obtain neworders, includes data fields (or “entities”) referred to asCustomerNameLast and CustomerNameFirst, it is a relativelystraightforward process to map those entries to an accounting softwareprogram having the data fields BillingAddressFirst andBillingAddressLast. In such an integration system, the relationshipbetween entities in one system (i.e., computer system or application)and entities in another system can be stored in tables. A systemadministrator can configure entity mapping between the systems byselecting between the various entities of the two systems.

[0008] When an integration system actually executes the transactionbetween the source application and the destination application asdefined by the mapping process, several steps occur. First the sourceapplication creates a document or message, which contains data stored asentities, and transmits or publishes that document or message to theintegration application. The integration application transforms thedocument or message according to the transformation processes definedduring the mapping. Then the integration application posts thetransformed message to the destination application.

[0009] Historically, in integration systems, entities in one businessapplication have been linked to entities in a second businessapplication by requiring each system to maintain a special field tostore the unique identifiers of the other business application. Thisapproach to maintaining and managing relationships between applicationspresents many problems. First the developer of the application needs theforesight to anticipate every possible field characteristic that canappear in another application program. Second this additional fieldrestricts the number of associated entities for each entity to one.Third this approach requires the application to have additional logicprogrammed in to allow the retrieval of that record by that identifier.

SUMMARY OF THE INVENTION

[0010] The present invention includes a linking system that assists inmanaging the linking of entities contained in messages between twobusiness applications. The linking system of the present invention alsoassists in the updating of entities between two business applicationswhen an entity is updated in one application.

[0011] Messages are picked up by a receive function from a message queueand processed by a preprocessor. While the message is in thepreprocessor, the message is processed to identify various features ofthe message necessary to post the message to the destination system.Once the preprocessing is finished the message is returned to thereceive function.

[0012] The message is then passed from the receive function to thechannels, and processed through channels in the integration server. Thechannels transform the message according to a predefined process. Thisprocess transforms the message from a format that is useable by thesource system to a format that is useable by the destination system.Once the message is transformed it is delivered to a pipeline.

[0013] When the message reaches the pipeline the message is posted tothe destination system. Following a successful posting of the message,the pipeline receives a confirmation back that the post was successful.Contained in this confirmation are associated parameters related to theentities created in the destination system. The pipeline then invokes alinker to establish links between related entities in source anddestination application programs in a link table. The linker includesseveral protocols that manage the entry of entities in the link tables.Using the protocols contained in linker, the linker takes informationfrom both application programs and creates two records in a first tableand one record in a second table that represents the links between thetwo records based upon their entries in the first table. The linkerestablishes these links in the table by calling on modules contained inthe linker.

[0014] Another feature of the present invention is a linker thatincludes a series of meta data tables that store information about thetwo systems or multiple systems that have information that is beingintegrated. The linker records for each entity in each system beingintegrated with entities in other system. It also includes severalmodules or APIs which allow for manipulation of the data and recordsstored in the tables. These modules includes a module for insertinglinks and entities into the tables, a module for removing links in thetables, a module for removing an entity and associated links from thetables, a module for bulk loading entities and links into the tables,and a module for reporting the entities that are linked in the tables.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015]FIG. 1 is a block diagram of one exemplary environment in whichthe present invention can be used.

[0016]FIG. 2 is a block diagram illustrating an exemplary networkenvironment in which the present invention can be implemented.

[0017]FIG. 3 is a block diagram illustrating two components of theintegration server illustrated in FIG. 2.

[0018]FIG. 4 is a block diagram illustrating the components of theorchestration engine illustrated in FIG. 3.

[0019]FIG. 5. is a block diagram illustrating the components of themessaging service illustrated in FIG. 3.

[0020] FIGS. 6 shows a block diagram illustrating a messaging serviceincluding a linker.

[0021]FIG. 7 is a flow chart illustrating the basic steps that areexecuted by the messaging service when a message is published to themessaging service.

[0022]FIG. 8. is a flow chart illustrating the steps executed by thestate management system when a document reaches the pipeline.

[0023]FIG. 9 is a flow chart illustrating the steps executed by theUpsertLink API.

[0024]FIG. 10 is a flow chart illustrating the step performed by thelinker when the Deletelink API is executed.

[0025]FIG. 11 is a flow chart illustrating the steps performed by thelinker when the DeleteInstance API is executed.

[0026]FIG. 12 is a flow chart illustrating the steps executed by thelinker when the GetDestinationLink API is executed.

[0027]FIG. 13 is a flow chart illustrating the steps executed by thelinker when the Load API is executed.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0028] The present invention deals with managing messaging systems.However, prior to discussing the invention in greater detail, oneembodiment of an environment in which the invention can be used will bediscussed.

[0029]FIG. 1 illustrates an example of a suitable computing systemenvironment 100 on which the invention may be implemented. The computingsystem environment 100 is only one example of a suitable computingenvironment and is not intended to suggest any limitation as to thescope of use or functionality of the invention. Neither should thecomputing environment 100 be interpreted as having any dependency orrequirement relating to any one or combination of components illustratedin the exemplary operating environment 100.

[0030] The invention is operational with numerous other general purposeor special purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

[0031] The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

[0032] With reference to FIG. 1, an exemplary system for implementingthe invention includes a general purpose computing device in the form ofa computer 110. Components of computer 110 may include, but are notlimited to, a processing unit 120, a system memory 130, and a system bus121 that couples various system components including the system memoryto the processing unit 120. The system bus 121 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. By way of example, and not limitation, such architecturesinclude Industry Standard Architecture (ISA) bus, Micro ChannelArchitecture (MCA) bus, Enhanced ISA (EISA) bus, Video ElectronicsStandards Association (VESA) local bus, and Peripheral ComponentInterconnect (PCI) bus also known as Mezzanine bus.

[0033] Computer 110 typically includes a variety of computer readablemedia. Computer readable media can be any available media that can beaccessed by computer 110 and includes both volatile and nonvolatilemedia, removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by computer 110. Communication media typicallyembodies computer readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. The term “modulated data signal” means a signal that has one ormore of its characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of any of the aboveshould also be included within the scope of computer readable media.

[0034] The system memory 130 includes computer storage media in the formof volatile and/or nonvolatile memory such as read only memory (ROM) 131and random access memory (RAM) 132. A basic input/output system 133(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 110, such as during start-up, istypically stored in ROM 131. RAM 132 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 120. By way of example, and notlimitation, FIG. 1 illustrates operating system 134, applicationprograms 135, other program modules 136, and program data 137.

[0035] The computer 110 may also include other removable/non-removablevolatile/nonvolatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, nonvolatile magnetic disk 152,and an optical disk drive 155 that reads from or writes to a removable,nonvolatile optical disk 156 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate ROM, and the like. The hard disk drive 141 is typically connectedto the system bus 121 through a non-removable memory interface such asinterface 140, and magnetic disk drive 151 and optical disk drive 155are typically connected to the system bus 121 by a removable memoryinterface, such as interface 150.

[0036] The drives and their associated computer storage media discussedabove and illustrated in FIG. 1, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 110. In FIG. 1, for example, hard disk drive 141 is illustratedas storing operating system 144, application programs 145, other programmodules 146, and program data 147. Note that these components can eitherbe the same as or different from operating system 134, applicationprograms 135, other program modules 136, and program data 137. Operatingsystem 144, application programs 145, other program modules 146, andprogram data 147 are given different numbers here to illustrate that, ata minimum, they are different copies.

[0037] A user may enter commands and information into the computer 110through input devices such as a keyboard 162, a microphone 163, and apointing device 161, such as a mouse, trackball or touch pad. Otherinput devices (not shown) may include a joystick, game pad, satellitedish, scanner, or the like. These and other input devices are oftenconnected to the processing unit 120 through a user input interface 160that is coupled to the system bus, but may be connected by otherinterface and bus structures, such as a parallel port, game port or auniversal serial bus (USB). A monitor 191 or other type of displaydevice is also connected to the system bus 121 via an interface, such asa video interface 190. In addition to the monitor, computers may alsoinclude other peripheral output devices such as speakers 197 and printer196, which may be connected through an output peripheral interface 195.

[0038] The computer 110 may operate in a networked environment usinglogical connections to one or more remote computers, such as a remotecomputer 180. The remote computer 180 may be a personal computer, ahand-held device, a server, a router, a network PC, a peer device orother common network node, and typically includes many or all of theelements described above relative to the computer 110. The logicalconnections depicted in FIG. 1 include a local area network (LAN) 171and a wide area network (WAN) 173, but may also include other networks.Such networking environments are commonplace in offices, enterprise-widecomputer networks, intranets and the Internet.

[0039] When used in a LAN networking environment, the computer 110 isconnected to the LAN 171 through a network interface or adapter 170.When used in a WAN networking environment, the computer 110 typicallyincludes a modem 172 or other means for establishing communications overthe WAN 173, such as the Internet. The modem 172, which may be internalor external, may be connected to the system bus 121 via the user inputinterface 160, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 110, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 1 illustrates remoteapplication programs 185 as residing on remote computer 180. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

[0040]FIG. 2 is a block diagram illustrating a network environment inwhich the present invention can be implemented. Network 200 includes asource system 210, a destination system 220, an integration server 230,and document transports 240 and 245. Source system 210 is, in oneembodiment, a computer, such as computer 110 or another system having anapplication program 211 that produces documents or messages 215. System210 can be remote from or local to integration server 230. Similarly,system 220 can be a computer having an application program 221 that canreceive documents or messages 215. Documents and messages aretransported to and from integration server 230 via transports 240 and245. Transports 240 and 245 can include hypertext transfer protocol(HTTP), secure hypertext transfer protocol (HTTPS), simple mailtransport protocol (SMTP), Microsoft Message Queue (MSMQ) or other knowntransport protocols. Of course, the transports can be bidirectional butare simply shown in a context in which system 210 is the source andsystem 220 is the destination, by way of example.

[0041] Application program 211 produces a document or message 215, suchas a sales order which is needed by application program 221. Messagesare illustratively documents that have additional information attachedto them, such as headers or footers which produce information regardingthe information contained in the document. However, because applicationprogram 211 and application program 221 reside on different computers,and may use different formats or structures for data in a document, thedocument 215 must be transformed or altered so that application program221 can read and understand the incoming document.

[0042] Integration server 230 provides this functionality by convertingor transforming the format of the document 215 from application program211 to the format of application program 221. Integration server 230 canbe a computer (such as computer 110 in FIG. 1), a network server or anyother information processing system. It should be noted that while onlyone integration server 230 is illustrated, systems 210 and 220 can beconnected through multiple servers 230. Further, messages 215 can bepassed through multiple servers 230 to reach the destination system 220.

[0043]FIG. 3 is a high level illustration of two components of theintegration server 230 of FIG. 2. In one embodiment integration sever230 is the BIZTALK Server from Microsoft Corporation of RedmondWashington. However, the integration sever 230 can be any other businessto business (B2B) server or B2B integration control component or anycomponent for messaging between two systems. The integration server 230includes two separate individual server components, a messaging service300 and an orchestration engine 350. However, other components can beincluded in server 230.

[0044] The orchestration engine 350 is, in one embodiment a COM+application that manages complex distributed processes that requiremultiple decisions, loops and actions, such as, for example, processingan insurance policy application. The orchestration engine 350 is asimplistic graphical means of expressing a business workflow to assistin instructing the integration server 230 in learning how to transfer adocument 215 from the format of application program 211 to the format ofapplication program 221. The orchestration engine 350 assists adeveloper in creating workflow diagrams, or orchestration schedules,through the use of an orchestration designer to visually define andbuild the business process necessary to convert the formats.

[0045]FIG. 4 is a block diagram illustrating the components of theorchestration engine 350. The orchestration designer 400 is a tool thatfacilitates the modeling process. One illustrative embodiment leveragesexisting graphical design software (such as VISIO 2000 also by MicrosoftCorporation or any other graphical design component) to provide agraphical user interface (GUI) to assist the developer in developing aprocess model for the transformation. The process flow is graphicallydisplayed to the developer as a flow chart 463. The orchestrationdesigner 400 includes process options such as action 410, decision 420,while 430, abort 440, fork 450 and join 460. However, additional ordifferent process options can be included in orchestration designer 400.Each of these process options allows for a different flow routine to beexecuted. For purposes of clarity and completeness a brief descriptionof some illustrative examples of these process options is provided belowfor the sake of discussion.

[0046] The action process option 410 defines a process that typicallyrequires information to be manipulated. For example, it might be anaction step that runs the information through a filter, or it might bean action step that reformats data being processed.

[0047] The decision process option 420 defines a process where a yes/no,data query, or other information analysis is conducted, and the resultdetermines which branch to follow to continue with the workflow. It maybe determined that if a field is blank to have a branch that enables anaction to fill the field with data. However, if the field is not blank,then a branch would allow the process to continue.

[0048] The while object process option 430 allows a process to loopuntil a specific event occurs. This might be a process that continuesuntil the end of a dataset is reached, or it might be a process thatcontinues until a specific data record is reached, for example. A branchcan then be followed to a subsequent decision or action.

[0049] The abort object process option 440 identifies a step where theprocess needs to be terminated. In many cases, the abort 440 becomes afail-safe test that determines that the information received ismalformed, signaling data corruption in the transmission or just thatincorrect data was transmitted. This step can provide a way to stop aprocess from continuing with incorrect information

[0050] The fork process option 450 splits a process to conductsimultaneous tasks that can then be joined later through the joinprocess option 460. This can, for example, be a process that enablesinformation to be gathered for decisions to be made. For example, a forkin a process can be created to gather price and availability informationfrom multiple vendors. This information can be gathered and processedsimultaneously. From this fork, when the information has been analyzed,the system can join back together to continue with the process.

[0051] These process options are then linked to each other to generate aflow chart 463 or flow process for the desired process. The flow chart463 is then compiled into a drawing 465. This “drawing” is referred toas an XLANG scheduling drawing 465. The XLANG scheduling drawing 465 isa version of flow chart 463 compiled into XLANG. XLANG is an XML basedworkflow definition language and is but one example of a definitionlanguage that can drive the orchestration engine 350.

[0052] Once the flow chart 463 has been converted into XLANG (or otherdefinition language) it is passed to the XLANG scheduler engine 470. TheXLANG scheduler engine 470 controls the activation, execution,dehydration, and rehydration of an XLANG schedule 465. This processtakes the final designed business and technical process and puts it intoa format that can be executed by the integration server 230.

[0053] The XLANG schedule 465 defines the steps to be performed, thecomponents that must be called, and the data that will pass through theintegration server 230 in order to complete the process generated by theorchestration designer 350. When documents or messages 215 are sent fromthe XLANG schedule 465 to the messaging service 300, the name of achannel that will receive the documents is defined. After the channel isdefined, documents can pass from the messaging service 300 to anorchestration service 480 through the channel.

[0054] In the orchestration service 480, the document 215 is eitherprocessed in XML format for instance, or if the document is not in XML,the XLANG scheduler engine 470 can embed the document 215 in theengine's 470 standard XML wrapper. However, other internal languageformats can be used. When inside the XLANG scheduler 470, the document215 can undergo any modifications defined by the process. Afterprocessing the document 215, it can then be sent back to messagingservice 300 for posting to a destination, or it can be sent out of theXLANG scheduler 470 to a private or public queue.

[0055] The XLANG scheduler engine 470 employs two processes to preventscaling problems caused by multiple XLANG schedules running in parallelover extended periods of time. These two processes are referred to asdehydration and rehydration. Dehydration involves taking a schedule 465that is not immediately required and writing the status of the scheduleto a status database. As the schedule 465 is not being processed at thattime, more resources are freed. Rehydration involves reloading theschedule 465 into the main memory of the server with the same status theschedule 465 had at the time of dehydration.

[0056] The orchestration service 480 includes ports 482, which are namedlocations in the XLANG schedule 465. The port 482 implements atechnology, such as COM, MSMQ, or the messaging service, in order tosend or receive messages. However, other technologies can be implementedas well. The port 482 can implement communications either synchronouslyor asynchronously, and is used to send messages or documents 215 to, orreceive messages from the XLANG schedule 465. Orchestration ports 482differ from messaging ports used in the messaging service 300. Messagingports will be described in greater detail in FIG. 5.

[0057]FIG. 5. is a block diagram illustrating the components ofmessaging service 300 in FIG. 3. Messaging service 300 includesmessaging objects 500, receive functions 550, and parsers 560. However,other components can be included in the messaging service 300 such asinterchange component 551 and queue 580. Messaging services 300 are acomponent in the integration server 230 that enable the sending,receiving, parsing, and tracking of messages and documents 215 fromoutside organizations or from applications, such as application programs211 and 221 in FIG. 2. In addition, messaging services 300 include theability to generate receipts for certain file formats, correlate and mapdata, verify the integrity of documents, and provide secure methods forexchanging documents with outside sources and applications.

[0058] Messaging objects 500, receive functions 550, component objectmodels (COM) and COM+methods, parsers 560, and SQL server databases(such as, for example, Microsoft SQL Server version 7.0 with SP2), canbe used to implement the messaging services 300. Messaging objects 500are used to configure the necessary properties to process and transmitdocuments submitted to the integration server 230. Receive functions550, and in some instances other methods, are used to submit incomingdocuments or messages 215 to the integration server 230 for processing.Once a document or messages 215 is submitted, the appropriate parser 560parses the document and, if necessary, converts the document to anotherformat (such as an XML format). Finally a tracking database 561 storesdocument records for both incoming and outgoing documents 215 that areprocessed by the integration server 230.

[0059] Messaging objects 500 include channels 510, messaging ports 515,distribution lists 520, organizations 525, document definitions 530, andenvelopes 535. However, other components can also be included. Theintegration server 230 uses these objects 500 to configure the necessaryproperties to process and transmit submitted documents 215 or messages.For purposes of this discussion the terms documents and messages areused interchangeably for data that is submitted to the integrationserver 230 for processing. The primary difference between documents andmessages is that messages contain additional information such as headersand footers that are not present in documents.

[0060] Channel 510 is a named entity that is used as a reception gatewayfor documents and messages from either a previously known originatingorganization or from an open messaging source. An open messaging sourceis a generic channel that is configured to identify the origin of themessage from the content of the message. In one embodiment, channel 510is bound to a single messaging port 515. Whereas, a messaging port 515can have an unlimited number of channels 510 assigned to it.

[0061] A channel 510 includes a set of properties, which identifies thesource organization or application that has sent out the document ormessage. The channel 510 also defines the specific steps that areperformed by the integration server 230 before the document is deliveredto the associated messaging port 515. The channel properties can includea source organization or application, a document definition, atransformation map, field and document tracking settings, and archivinginformation. Only properly formatted and verifiable data is sent throughchannel 510. In order to ensure this, the channel 510 defines the formatin which it expects to receive messages, and also defines how this datawill be forwarded through to the associated messaging port 515. Both theinbound and outbound document definitions are references to documentspecifications that are expressed in a format that is understandable bythe integration server 230. When a document 215 is submitted to thechannel 510, it is verified against the inbound specification. If thedocument is not compatible with the required format for the channel 510,the data is rejected and the message is placed in a suspended queue. Thequeues 580 will be discussed in greater detail below.

[0062] When a document 215 is in the channel 510 the document 215 canillustratively be altered in two ways by transformation or translation.The document 215 is altered such that the definition of the document,and the data between records and fields for the source documentspecification, and the records and fields for destination documentspecification are correlated. This process of correlating informationfrom one input format to another goes through one of two differentprocesses. The integration server 230 can alter the schema of theinformation in a process that is said to transform the information (orbe a transformation), or the integration server 230 can actually alterthe data itself in a process that is said to be a translation of theinformation.

[0063] In one embodiment, the transformation process used by theintegration server 230 is handled by script coding using functoids. Afunctoid is a reusable function built-in to the orchestration serviceengine 480 that enables simple as well as complex structuralmanipulation between source and destination specifications. Atransformation of information is common because input and output formatsfrequently do not match, and need to be altered so that fields from oneformat match the fields of the other format during the transformationprocess. The translation process, although by definition alters the datain the document 215, does not necessarily delete the information andreplace it with completely different data, but instead the informationmay be analyzed and modified for a new or different format. For example,data translation can replace a “State” definition such as “Calif” with astandard two-letter “CA” postal format. This transformation of thedocuments occurs in the channel 510 as the document 215 is passedthrough the messaging service 300.

[0064] Messaging ports 515 are a set of properties that specify how amessage or document 215 is transported to a destination organization orapplication. Messaging port properties can include transport services,destination organization or application, security settings, and envelopesettings.

[0065] Each messaging port 515 has a primary transport 516 and anoptional backup transport 517. These transports 516 and 517 are used forbinding communications endpoints that link to a remote organization orto an application. The messaging ports 515 provide a map between anabstract addressing scheme of organization identifiers and transportdependent addresses through channels 510. Hence, a multitude ofmessaging ports 515 can exist for a single destination organization orapplication.

[0066] The primary and back-up transports 516 and 517 are push servicesthat use transport protocols supported by integration server 230 todeliver documents or messages 215 to the destination application. Thesetransport protocols can include hypertext transfer protocol (HTTP),secure hypertext transfer protocol (HTTPS), simple mail transportprotocol (SMTP), Microsoft Message Queue (MSMQ) or other transportprotocols. Further, special transport protocols can be used such asApplication Integration Components (AIC) , which allow documents andmessages 215 to be submitted directly, or to be carried over proprietaryprotocols.

[0067] When a document 215 is sent to the destination application 221identified by the messaging port 515, the integration server 230 usesthe primary transport 516. The back-up transport 517 is used only whenthere is a transport level error, such as an unreachable or crashedserver when multiple servers are used. Therefore, the primary andback-up transports 516 and 517 generally use the same applicationprotocols, but point to different channels 515 to increase robustness ofthe system.

[0068] Distribution lists 520 are a group of messaging ports 515.Distribution lists 520 are used to send the same document 215 to morethan one organization or application. The distribution list 520 issometimes referred to as a port group.

[0069] Organizations 525 are trading partners or other entities withwhom the integration server 230 exchanges messages and documents. Theseorganizations can be internal to the server, such as an application ordivision within the host organization, or can be external to the server,such as an external business or an application at that externalbusiness.

[0070] Document definitions 530 are a set of properties that representsan inbound or an outbound document 215. The document definition 530 caninclude a property that provides a pointer to a document specification.A document specification 531 defines the document structure, documenttype, and version. However, a pointer from the document definition 530to the document specification 531 is not required.

[0071] One or more documents 215 can be packaged together as aninterchange, which is submitted to the integration server 230. Theinterchange can be a single document along with header and footerinformation. However, the interchange can also involve multipledocuments having header and footer information. As headers and footersare used in the interchange, the messaging service 300 needs to be ableto identify where in the interchange the document begins and ends.Depending on the file format the messaging service 300 will need to knowhow the header and footer information is structured, and hence where thedocument 215 is found in the interchange. The messaging service 300receives this information through the use of envelopes 535.

[0072] Envelopes 535 are a set of properties that represent thetransport information for a document. The envelope 535 wraps data fortransport and selects the destination of the data. Envelopes 535 includeheaders for messages and function groups. However, other types ofheaders can be included in the envelope 535. In one embodiment, themessaging service 300 uses six types of envelopes 535. These are X12,EDIFACT, Flat-File, Custom, Reliable, and CustomXML. However, other oradditional types of envelopes can be used. The type of envelope 535 usedto wrap the document 215 tells the messaging service 300 which parser560 should be used to process the incoming document. An envelope 535associated with an inbound message or document provides the integrationserver 230 with the information that the server needs to interpret thesubmitted document. For example, the envelope 535 can include a pointerto a document definition 530. Envelopes 535 associated with an outboundmessage or document give an outside server the information it needs tocreate the document.

[0073] Inbound messages do not necessarily require an envelope 535 toaccompany the document 215. Envelopes 535 are required only fordocuments that are submitted to the messaging service 300 that are in aformat that the messaging service 300 cannot able to readily recognize.However, for outbound documents messages an envelope 535 may berequired. The envelope 535 is created during the configuration of theoutbound messaging port 515. This allows the messaging service 300,through a serializer, to build a complete outgoing message including theheader and footer information, so that when the document 215 issubmitted to another messaging service it is able to identify the typeof documents being submitted, regardless of the internal format used bythe destination server.

[0074] Message and documents are submitted into the integration server230 through an interchange component 551 or a receive function 550. Theinterchange component 551 is configured to support transactions. As aresult, the interchange component 551 adopts the characteristics of theapplication that is invoking the interchange component 551. Once amessage or document is submitted, the interchange component 551 selectsthe appropriate parser 560 in integration server 230 based oninformation in the envelope 535 to parse the message or document, unlessthe message or document is submitted with a pass-through flag enabled.The parser 560 converts the document 215 contained in the message fromits native format to a format that is understandable by the messagingservice of the server. In one example, if the integration server 230 isconfigured to operate in an XML format, the parser does not parsedocuments that are submitted in XML. The integration server 230 alsodoes not parse messages and documents submitted with the pass-throughflag enabled, but passes the document on to the destination in theformat it was submitted.

[0075] Receive functions 550 are components of the integration server230 that monitor either a message queue 580 or a directory in a filesystem directory. When a message arrives in the queue 580 or a file isplaced in the directory, the receive function 550 takes the file ormessage and submits it directly for processing by the messaging service300. File receive functions watch a certain directory identified on thelogical drive or on a universal naming convention (UNC) file path. Thefile receive function continuously monitors the identified directory forfiles that have specified file extensions, and when files having theseextensions arrive, the file receive function submits the associateddocument for processing. Message queue receive functions monitor queuesin the shared queue database 580 and submits arriving items into thequeue for processing.

[0076] When a message or document 215 is submitted to the integrationserver 230, the document or message 215 is stored in a shared queuedatabase 580 until it is picked up for processing. The shared queue 580includes four separate queues, a work queue 581, a scheduled queue 582,a retry queue 583, and a suspended queue 584. By accessing these queues580, it is possible to determine what stage of processing a message ordocument 215 is in. For example, it is possible to determine if adocument has been processed and is waiting for transmission or if themessage or document failed processing. Messages and documents 215 appearin each of the queues in the order of “first in, first out.” That is,the oldest items in the work, retry, scheduled, or suspended queueappear first and the newest items appear last.

[0077] The work queue 581 holds all messages and documents that havebeen submitted to the integration server 230 for asynchronousprocessing, but have not been picked up by the receive function 550. Ifmessaging processing is synchronous, the work queue 581 is not used andthe document is processed immediately. The work queue 581 containsmessages and documents 215 that are currently waiting to process.Messages and documents placed in the work queue 581 are processed uponarrival, and therefore do not remain in the work queue for long periodsof time.

[0078] The receive function 550 polls the work queue 581 atpredetermined time intervals to pickup the next documents in the workqueue 581 for processing. These documents are picked up by the receivefunction 550 as batches of documents or messages. The rate of release ofdocuments in the work queue 581 depends on the configuration of theintegration server 230. Any message in the work queue 581 can be movedto the suspended queue 584. Once a message or document is moved to thesuspended queue 584, it can be deleted, resubmitted, or retransmitted tothe work queue 581 to complete processing at a later time.

[0079] When a message or document is processed, a channel 510 and amessage port 515 are associated with the message. The message portdefinitions can include a service window for processing the message. Forexample, the service window could be defined as between Midnight and1:00 am. If the message is processed outside this service window themessage is redirected to the scheduled queue 582. The scheduled queue582 contains messages and documents that have been processed by theintegration server 230 and are waiting for transmission based on theservice window. The scheduled queue 582 holds the message until theservice window opens. Once the service window is opened the messageservice 300 attempts to resend the message through the previouslyidentified message port 515. Like the work queue 581, any item in thescheduled queue 582 can be moved to the suspended queue 584.

[0080] If a message fails to send or post due to a communications erroror for other reasons the message is placed in the retry queue 583. Theretry queue 583 contains messages and documents 215 that are to beresubmitted for delivery and documents that are waiting for reliablemessaging receipts. A message can be sent to the retry queue 583 ifduring processing a receipt was expected, but was not present. When areceipt is required, a message can remain in the retry queue 583 untilthe receipt is received, or it can be treated like all other messages inthe retry queue 583 and automatically resent. The message will continueto be resent according to the frequency and number of attemptsconfigured by the channel 510. If the number of attempts to resend themessage exceeds a threshold or the document's time to live (TTL) isexceeded, the message 215 is moved to the suspended queue 584.

[0081] The suspended queue 584 stores and displays messages anddocuments 215 that have failed processing, or that have failed to postto their destination. Messages can fail because some fatal unrecoverableerror occurred in processing. For example, these errors can includeparsing errors, serialization errors, and missing channels. Messages inthe suspended queue 584 have an associated error report that indicateswhy the message was placed in the suspended queue 584. Depending on thenature of the failure most messages or documents in the suspended queue584 can be deleted, resubmitted, or retransmitted to the server 230 forprocessing. However, in one embodiment, some messages and documentscannot be resubmitted, for example, messages that failed parsing cannotbe resubmitted or retransmitted.

[0082] The integration server 230 takes documents 215 from the queues580 in batches for processing and transformation through the receivefunction 550. However, once documents are taken from the queues 580 inbatches, any order that may have existed in the documents is lost. It isonly by chance that documents and messages 215 are processed in thecorrect order. To help ensure that documents 215 are posted in thecorrect order, the integration server 230 also includes a documentmessaging state management engine that assists in posting a documentfrom a first application program 211 to a second application program 221in an order that is sequentially correct for the second applicationprogram 221. The document messaging state management engine isillustratively a component module of the messaging service 300.

[0083]FIG. 6 illustrates a messaging service 600 including the linkingsystem of the present invention. Messaging service 600 includes apipeline 610, channel 620, a first application receive function 630, asecond application receive function 640, a first application or sourceapplication integration component (AIC) 635, a second application orsource destination AIC 645, a first application queue 633, a secondapplication queue 643, a preprocessor 660 and a linker 650. Thecomponents of linker 650 are illustrated in greater detail in FIG. 8.The messaging service 600 is connected to a first application program631 and a second application program 641.

[0084]FIG. 7 is a flow diagram illustrating the basic steps that areexecuted by the messaging service 600 when a message is published to themessaging service 600. FIGS. 6 and 7 will be described together andreferences made to each drawing.

[0085] The first application program 631 publishes a message 215 tomessaging service 600. The messaging service 600 receives the message215 and places the message 215 in the work queue of the queues 633.Depending on the characteristics of the message 215, the message 215 canbe placed in other queues in the queue 633. The publishing of themessage 215 from first application program 631 to the queue 633 isillustrated at block 710.

[0086] When the message 215 is placed in one of the queues 633 thereceive function 630 pulls the message 215 from the associated queue 633as one of many messages in a batch 636. The receive function 630 thenpasses each message 215 in the batch 636 to preprocessor 660. Thispassing of the message to the preprocessor is illustrated at block 720.

[0087] At block 730, preprocessor 660 proceeds to preprocess eachmessage 215 provided by the receive function 630. Preprocessor 660analyzes the contents of each message including any header and footerinformation contained in the message, and determines which channel 610and messaging ports are needed for message 215 to post to thedestination application program 641. While the message is inpreprocessor 660 other transformations can occur. For example, themessaging service 600 can included a state management system, such asthe state management system disclosed in application Ser. No.10/402,337, in order to maintain correct message ordering and postingpriorities. These additional procedures can be performed while themessage 215 is in the preprocessor 660.

[0088] Also, while the message is in the preprocessor 660, preprocessor660 can make changes to the message's header and footer so that it iscompatible with the format of the identified channels 610 of messagingservice 600. Once the preprocessing is completed the preprocessor 660passes the message 215 back to the receive function 630. The providingof the message 215 to the receive function 630 is illustrated at block740.

[0089] Once the message 215 is returned to the receive function 630, themessage 215 is then passed to one of the channels 620 identified by thepreprocessor 660. When the message 215 is in the channel 620 the messagecan be transformed according to the method developed in theorchestration service of FIG. 3 where the XLANG schedule converts themessage 215 from the source application's 631 format to the destinationapplication's 641 format. During this transformation process a check forany required external links can be performed. If the link is notrequired to exist then this check is not performed. If a link isrequired to exist and the link does not exist the system 600 can sendthe message to one of the queues of queue 633. This step is illustratedat block 750 of FIG. 7.

[0090] After the channel 620 processes the messages 215, the message 215is passed to the pipeline component 610 at block 755. The pipelinecomponent 610 is an interface that receives an indication that thetransformation and posting of a message or message 215 was successful.Once the message 215 is received by the pipeline 610, the transformedmessage 215 is then posted to the second application program 641. Forexample, if the message 215 was a create message to create a new entryin the destination application 641, the pipeline component 610 will postthe create message and associated information to destination applicationprogram 641. This posting of the message 215 is illustrated at block760.

[0091] Following the posting of the message 215 to the destinationapplication program 641, the pipeline 610 receives from the destinationapplication program 641 the results of the posting of the message atblock 765. The results indicate if the post was successful. For example,in the above example of creating a new record in application program641, the pipeline 610 waits for an indication that the post wassuccessful. This is illustrated at block 770. If the results of theposting are unsuccessful the pipeline 610 would submit the message tothe retry queue of queue 633 at block 790. If however the results of theposting are successful, the pipeline 610 invokes the linker 650 asillustrated by block 780.

[0092] When the linker 650 is invoked links between the related entitiesin both application programs 631 and 641 are established in a linktable. Linker 650 includes several protocols that manage the entry ofentities in the link tables. Using the protocols contained in linker650, the linker 650 takes information from both application programs 631and 641 and creates two records in a first table (one for each program)and one record in a second table that represents the links between thetwo records based upon their entries in the first table. These recordstell the service 600 that the two records are linked together. Theoperation of modules and of the linker tables will be discussed ingreater detail below. Also, depending on the nature of the message 215when the linker 650 is invoked links can be removed from the link tablescontained in linker 650. This establishment or removing of linksillustrated at block 785 of FIG. 7.

[0093] In the B2B environment there are often two separate businesssoftware applications that communicate to each other and integrate datawith each other. It is desirable to know when data is moved from onesystem to another system whether the second system made any changes tothat data that should be mapped back to the original or source system.However, commonly, these two separate systems are ignorant of theexistence of the other system. The linker 650 provides a mappingframework between the two systems regardless of the system's knowledgeof other systems. Linker 650 includes a series of meta data tables thatstore information about the two systems or multiple systems that haveinformation that is being integrated. Linker 650 includes records foreach entity in each system being integrated with entities in othersystem. FIG. 8 illustrates linker 650 in greater detail. Alsoillustrated in FIG. 8 are two exemplary systems 810 and 820 that arebeing linked or integrated with each other.

[0094] Linker 650 includes an integration entity instance table 830, anentity instance link table 840, a history or archive table 865 and APIs850. Systems 810 and 820 include messages which are represented asentities. An entity is a message from a specific area of the system 810or 820. For example, a back office entity could be inventory orcustomers. Each entity in system 810 and 820 is identified by a systemid 812 or 822 and a “friendly name” 814 and 824. The “friendly name” 814and 824 is a name that is easily understood by the user of the system810 or 820. The system id 812 and 822 is a numerical identifier used bythe systems 810 and 820 to identify the entity. In one embodiment thissystem id is a GUID.

[0095] Integration entity instance table 830 includes four columns. Aunique id column 831, a system name column 832, an entity id column 833and an entity key column 834. Unique id 831 is a unique internal id foreach entity identified in systems 810 and 820. The system id in column832 holds the system id identified by the linker 650 for the entity ofeither system 810 or 821. Entity id column 833 includes the unique idthat identifies the entity to the source system 810 or 820. Entity keycolumn 834 stores the value for the entity identified by the entity id832. However, other entries can be used in instance table 830. Thespecific format of the entity key in the parent system does not matteras the instance table 830 stores the value of the entity key in itsoriginal format. This also holds true for the name given to the entitykey by the destination system.

[0096] Entity instance link table 840 is a table that includes recordsor entities that are linked together. Link table 840 includes threeentries, a unique id entry 841, a first entity id entry 842 and a secondentity id entry 843. First id entry 842 is the unique id entry 831 fromthe integration entity table 830 for the first entity linked. Second identry 843 is the unique id entry 831 from integration entity instancetable 830 for the second entity linked. These two entries are identifiedin the entity instance link table 840 with the unique id 841.

[0097] History or archive table 865 is a table that includes entries andrecords in the integration entity instance table 830 that have beenremoved or deleted from table 830. History or archive table 860 containsthe identical information for the entity entry in integration entityinstance table 830.

[0098] Linker 650 includes several modules or APIs which allow formanipulation of the data and records stored in tables 830 and 840. Thesemodules include an Upsertlink API module 852, a Deletelink API module854, a GetDestinationLink API module 858 and a Load API module 856. TheUpsertlink API 852 is a module called for placing or entering newentities into the integration entity instance table 830 and for creatingnew links in the entity instance link table 840. The Deletelink API 854provides the linker 650 the ability to remove either a link record inthe entity instance link table 840 and/or an entry from the integrationentity instance table 830. The GetDestinationLink API 858 is a modulethat returns a collection of entities that are linked to a given systementity instance. The load API 856 is a bulk loading module for updatingor creating the integration entity link table 840, and creating anyinstance records in the integration entity instance table 830 that doesnot exist. Each of these APIs will be discussed in further detail withreference to FIGS. 9-13.

[0099]FIG. 9 is a flow diagram illustrating the steps executed by theUpsertLink API. When linker 650 is invoked to create a link between twoentities in two systems the linker 650 calls the UpsertLink API tocreate the necessary components in the linker tables 830 and 840. Theseparameters include the system ID's for both entities, the entity id'sfor both entities, and the entity keys for both entities.

[0100] When the UpsertLink API is called the program first gathers thedatasource connection. During this step the UpsertLink API gathers theparameters for each of the entities in each of the systems that are tobe linked. This gathering is illustrated at block 900. If the datasourceconnections cannot be found, for example, one of the requested entitiesdoes not exist in the system, the UpsertLink API will throw an exceptionto an error log. This step is illustrated at blocks 910 and 915.

[0101] If the datasource connections are found at block 910 theUpsertLink API retrieves the configuration information for each of thegathered entities. These configurations include linking characteristicsof the two entities. This step is illustrated at block 920. Followingthe retrieval of the configurations the UpsertLink API checks to see ifthe system ID obtained for each entity is valid. A valid system ID caninclude a GUID that refers to a system that is currently in use by themessaging service 600. This step is illustrated at block 930. If thesystem ID is not valid the UpsertLink API throws an exception or anerror at block 935. Next the UpsertLink API checks if the entity ID isvalid. This is illustrated at block 940. However other Ids can bechecked at blocks 930 and 940. If the entity ID is not valid theUpsertLink API throws and exception or an error at block 945. If anerror is thrown by the UpsertLink API during the process of checking thesystem ID, the entity ID, or during the gathering of the datasourceconnections, the UpsertLink API stops all further processing of theentities, and makes no entry into the link tables 830 and 840.

[0102] Following the verification that the system IDs and the entity IDsare valid the UpsertLink API determines if each passed in entity alreadyexists in the integration entity instance table 830. This check isillustrated at block 950. If the entity exists in the integration entityinstance table 830 the UpsertLink API identifies the record for theentity, and determines if there are any links pre-existing for theidentified record. This is illustrated at block 960. If a link is foundfor the passed in entity the UpsertLink API updates any records thathave the same entity ID as the passed in entity. This is illustrated atblock 970. If there is no preexisting link between the passed inentities the UpsertLink API creates a record for the entity in theentity instance linking table 840 by placing the unique ID 831 fromtable 830 for each entity in either column 841 or 842 of link table 840.This is illustrated at block 990. If the passed in entity does not existin the integration entity instance table 830 the UpsertLink API insertsthe entity into the table 830. In the table 830 the UpsertLink APIcreates an unique ID for the entity in column 831, enters the system IDfor the entity in column 832, enters the entity ID in column 833, andenters the entity key in column 834. This is illustrated at block 980.Then the UpsertLink API creates a record for the entity in the entityinstance linking table 840 by placing the unique ID 831 from table 830for each entity in either column 841 or 842 of link table 840 at block990.

[0103] An example of the process illustrated by FIG. 9 is describedbelow. Assuming that the messaging service 600 is integrating a CRMprogram (system A) with a Back Office account management program (systemB) . For example, System A has an entity type of customer with an entitykey of AARON, and System B has an entity type of account and an entitykey of a GUID, where these two entities are to be linked. Once theUpsertLink API has verified that the system ID's and entity ID's for theentities are valid, the UpsertLink API determines if the entity ofsystem A exists in the instance table 830. Assuming that this entitydoes not exist in instance table 830 the UpsertLink API generates aunique ID 831 for the entity and inserts the paramaters of the entity ofsystem A into the instance table 830. This entry for the entity into theinstance table 830 is illustrated at line 835 of FIG. 8. The UpsertLinkAPI repeats this process for the entity of system B. This entry of theentity into the instance table 830 is illustrated by the entry on line836 of FIG. 8. Following the entry of the two entities into the instancetable 830 the UpsertLink API then creates an entry in link table 840 forthe two entities. The entry for this link in table 840 is illustrated atline 845 of FIG. 8. As discussed above the entry in table 840 is theunique ID's 831 from table 830 for each entity. It should be noted thatit does not matter to the linker 650 if the entry ID for system A isbefore or after the entry for system B.

[0104]FIG. 10 is a flow diagram illustrating the steps performed by thelinker 650 when the Deletelink API is executed. The Deletelink API issimilar to UpsertLink API discussed in FIG. 9, except that theDeletelink API removes the links in link table 840, and can removeinstance records form the instance table 830.

[0105] When the Deletelink API is called the program first gathers thedatasource connection. During this step the Deletelink API gathers theparameters for the entities from each of the systems having entitiesthat are to be linked. This step is illustrated at block 1000. If thedatasource connections cannot be found, for example, one of therequested entities does not exist in the system, the Deletelink API willthrow an exception to an error log. This step is illustrated at blocks1010 and 1015.

[0106] If the datasource connections are found at block 1010, theDeletelink API retrieves the configuration information for each of thegathered entities. These configurations include the linkingcharacteristics of the two entities. This step is illustrated at block1020. Following the retrieval of the configurations the Deletelink APIchecks to see if the system ID obtained for the entity is valid. A validsystem ID can include a GUID that refers to a system that is currentlyin use by the messaging service 600. This step is illustrated at block1030. If the system ID is not valid the Deletelink API throws anexception or an error at block 1035. Next the Deletelink API checks ifthe entity ID is valid. This step is illustrated at block 1040. Onceagain other IDs can be checked at blocks 1030 and 1040. If the entity IDis not valid the Deletelink API throws and exception or an error atblock 1045. If an error is thrown by the Deletelink API during theprocess of checking ID's or during the gathering of the datasourceconnections the Deletelink API stops all further processing of theentities, and does not proceed to remove the link for the link table840.

[0107] Following the verification of the system ID and the entity ID theDeletelink API determines if there is an integration entity instancerecord in the instance table 830 for each entity. This step isillustrated at block 1050. If no matching record is found in theintegration entity instance table Deletelink API stops any furtherprocessing. This is illustrated at block 1055.

[0108] If the record is found the Deletelink API identifies all of thelinks in link table 840 having the unique ID 831 of the instance table830 for the submitted entities. This step is illustrated at block 1060.The Deletelink API then deletes each entry in the link table 840 foreach record in the table 840 that matches the submitted entities. Thisdeletion is illustrated at block 1070. Following the deletion of thelinks in the link table 840, the Deletelink API then searches the linktable 840 for each entity submitted, and checks if each of the submittedentities is linked to another entity (i.e. does the entity appear inanother record separate from the submitted entities), at block 1080. Ifone of the submitted entities is not linked to another entity in table840 the Deletelink API removes that entity's record from the integrationentity instance table 830. This step is illustrated at block 1085. Whenan entity is removed from the integration entity instance table, a copyof the entry's record is made in the history table 860. This step isillustrated at block 1090.

[0109]FIG. 10A is a flow diagram illustrating the steps performed by thelinker 650 when the Breaklink API is executed. The Breaklink API issimilar to Deletelink API discussed in FIG. 10, except that while theBreaklink API removes the links in link table 840, and it can onlyremove instance records form the instance table 830 if a specificinstruction to remove the instance is provided. For purposes ofsimplicity of the discussion, common elements or steps between theBreaklink API and the DeleteLink API will not be discussed below, andwill be referred to with the same reference numbers.

[0110] When the Breaklink API is called the program first gathers thedatasource connection. During this step the Breaklink API gathers theparameters for the entities from each of the systems having entitiesthat are to be linked. One of the parameters that are passed into theBreaklink API is an indication that one or both of the instancescontained in the link are to be deleted from the instance table 830. Forexample, this indication can be a true/false flag where true indicatesthat the instance can be removed. This step is illustrated at block1000. If the datasource connections cannot be found, for example, one ofthe requested entities does not exist in the system, the Breaklink APIwill throw an exception to an error log. This step is illustrated atblocks 1010 and 1015.

[0111] If the record is found the Breaklink API identifies all of thelinks in link table 840 having the unique ID 831 of the instance table830 for the submitted entities. This step is illustrated at block 1060.The Breaklink API then deletes each entry in the link table 840 for eachrecord in the table 840 that matches the submitted entities. Thisdeletion is illustrated at block 1070. Following the deletion of thelinks in the link table 840, the Breaklink API then searches the linktable 840 for each entity submitted, and checks if each of the submittedentities is linked to another entity (i.e. does the entity appear inanother record separate from the submitted entities), at block 1080.Next the Breaklink API checks the passed in flags (or other indication)to determine if the instance can be removed. This is illustrated atblock 1081. If one of the submitted entities is not linked to anotherentity in table 840, and the flag indicates that the entity can beremoved, the Breaklink API removes that entity's record from theintegration entity instance table 830 by invoking the DeleteInstance APIdiscussed below. This step is illustrated at block 1085. As discussedabove, when an entity is removed from the integration entity instancetable, a copy of the entry's record is made in the history table 860.This step is illustrated at block 1090.

[0112]FIG. 11 is a flow diagram illustrating the steps performed by thelinker 650 when the DeleteInstance API is executed. The DeleteInstanceAPI is similar to the Deletelink. API except that the DeleteInstance APIonly deletes the entity record and associated links for the identifiedentity. Blocks 1100 to 1145 of FIG. 11 are identical to blocks 1000 to1045 of FIG. 10 and represent similar processes.

[0113] Following the verification of the system ID and the entity ID forthe submitted entity, the DeleteInstance API determines if there is anintegration entity instance record for the entity. This step isillustrated at block 1150. If a record is found for the entity, theDeleteInstance API searches the link table 840 and identifies all of thelinks in link table 840 having the unique ID 831 of the submittedentity. This step is illustrated at block 1160. The DeleteInstance APIthen deletes each entry in the link table 840 that includes thesubmitted entity. This step is illustrated at block 1170. In analternative . embodiment, the DeleteInstance API only deletes theinstance from the instance table if the instance has no link entries inthe link table 840. Following the deletion of the records in the linktable 840, the DeleteInstance API deletes the associated record for thesubmitted entity in the instance table 830 at block 1180. At the timethe entity record is deleted from the integration entity instance table830, a copy of the entry record is made in the history table 860. Thisstep is illustrated at block 1190. However, unlike the DeleteLink APIthe DeleteInstance API does not check to see if the other entities havelinks remaining in the link table 840.

[0114]FIG. 12 is a flow diagram illustrating the steps executed by thelinker 650 when the GetDestinationLink API is executed. TheGetDestinationLink API is a module that returns to the user or othercomponent of the messaging service 600, a collection of entities thatare linked to the submitted entities. The GetDestinationLink API isinvoked when an update occurs to one entity and the system desires toupdate all entities linked to the updated entity. When theGetDestinationLink API is invoked by the linker 650 the API gathers thedatasource connection for the submitted entity. This is illustrated atblock 1200. If the API cannot find the datasource connection of thesubmitted entity, at block 1210, the GetDestinationLink API throws anexception at block 1215. If the GetDestinationLink API located thedatasource connection, the configuration information for the datasourceconnection is retrieved. This step is illustrated at block 1220.

[0115] Once the configuration information is retrieved, theGetDestinationLink API checks that the source system ID and thedestination system ID are valid at blocks 1230 and 1232. TheGetDestinationLink API also checks the validity of the entity types forboth the source and destination. These steps are illustrated at blocks1240 and 1242, respectively. If the IDs or the entity types are notvalid the GetDestinationLink API throws an exception at blocks 1234 and1244, respectively. The GetDestinationLink API then retrieves the uniqueID 831 of the source entity from the integration entity instance table830. This is illustrated at block 1250. If this unique ID 831 is notfound the GetDestinationLink API throws an exception at block 1254. Thisprocess is repeated for the entity of the destination system at block1252.

[0116] Following the retrieval of the unique ID from the instance table830 the GetDestinationLink API searches the link table 840 andidentifies all records having the unique ID 831 of the submitted sourceentity. The GetDestinationLink API also identifies all records havingthe unique ID 831 for the destination entity. This gathering of uniqueIds is illustrated at block 1260. The GetDestinationLink API thenreturns to the user the results of the search. This is illustrated atblock 1270. If no entities are found in the link table during the searchthe GetDestinationLink API returns as results of the search a defaultlink value, that is stored in the instance table 830. If the defaultlink value is not present then the API returns no value for the results.

[0117] The GetDestinationLink API can also be invoked in an overloadedconfiguration. In the overloaded configuration the GetDestinationLinkAPI is provided with less than all of the parameters in the instancetable 830. For example, the GetDestinationLink API can be provided withthe system ID and entity type. The GetDestinationLink API will thenidentify at blocks 1250 and 1252 those unique IDs 831 in the instancetable 830 that match the two parameters provided. Then, at block 1270,the API will return as results all entries that match the identifiedunique IDs 831.

[0118]FIG. 13 is a flow diagram illustrating the steps executed by thelinker 650 when the Load API is invoked. The Load API is a basic bulkload module that performs the same function as the UpsertLink API.However, the Load API allows for multiple entities and links to beentered in the tables 830 and 840 with one call. The entities that areto be linked are provided in a schema that is readable by the Load API.In one embodiment the schema is an XML schema.

[0119] The Load API receives the schema containing the sets of entitiesto be linked in link table 840 at block 1300. For each link that is tobe created, the Load API validates each of the entities that are to belinked. This validation is illustrated at block 1310. If the entitiescannot be validated an error is thrown at block 1314, and an entry ismade in a linkresults file. The Load API repeats this validation processuntil all of the sets of entities in the schema have been validated.This is illustrated at blocks 1316 and 1318.

[0120] Once all of the sets of entities in the schema have beenvalidated, the Load API calls the UpsertLink API for each set ofentities that were sucussfully validated. This calling of the UpsertLinkis illustrated at block 1320. During the UpsertLink call, the Load APIchecks for any business exceptions that may occur, and checks for anysystem exceptions that may occur. This check is illustrated at block1330. However, other exemptions can be checked at block 1330.

[0121] If a business exception occurred the Load API adds a result ofthe exception to the linksresult file. This is illustrated at blocks1340 and 1345. Following the reporting of any business exception theLoad API advances to the next set of entities to be linked at blocks1360 and 1365. If a system exception occurs, at block 1350, the Load APIadds a system error to the linkresults file at block 1352. Then the LoadAPI stops processing the links in the schema at block 1354, and returnsthe linksresults file to the user at block 1356. Then the Load APIthrows an exception to the linker 650 at block 1358 to stop the linker650.

[0122] Following the successful processing of the schema, the Load APIreturns to the user the contents of the linksresults file to the user atblock 1370. If there were no business exceptions or validity problemsthe Load API returns an empty linksresult file to the user. If therewere exceptions, then the user can view the contents of the file andcorrect any problems identified.

[0123] In summary, the present invention is directed to a linking systemthat assists in managing the linking of entities contained in messagesbetween two business applications. The linking system of the presentinvention also assists in the updating of entities between two businessapplications when an entity is updated in one application.

[0124] Messages are picked up by a receive function from a message queueand processed by a preprocessor. While the message is in thepreprocessor, the message is processed to identify various features ofthe message necessary to post the message to the destination system.Once the preprocessing is finished the message is returned to thereceive function.

[0125] The message is then passed from the receive function to thechannels, and processed through channels in the integration server. Thechannels transform the message according to a predefined process. Thisprocess transforms the message from a format that is useable by thesource system to a format that is useable by the destination system.Once the message is transformed it is delivered to a pipeline.

[0126] When the message reaches the pipeline the message is posted tothe destination system. Following a successful posting of the message,the pipeline receives a confirmation back that the post was successful.Contained in this confirmation are associated parameters related to theentities created in the destination system. The pipeline then invokes alinker to establish links in a link table between related entities inthe source and the destination application programs. The linker includesseveral protocols that manage the entry of entities in the link tables.Using these protocols the linker takes information regarding theentities from both application programs, and creates two records in afirst (instance) table and one record in a second (link) table thatrepresents the links between the two records based upon their entries inthe first table. The linker establishes these links in the table bycalling on modules contained in the linker. The linker can also updatethe records in the first table if a record previously existed for theentity.

[0127] Another feature of the present invention is a linker thatincludes a series of meta data tables that store information about thetwo systems or multiple systems that have information that is beingintegrated. The linker records for each entity in each system beingintegrated with entities in other system. It also includes severalmodules or APIs which allow for manipulation of the data and recordsstored in the tables. These modules includes a module for insertinglinks and entities into the tables, a module for removing links in thetables, a module for removing an entity and associated links from thetables, a module for bulk loading entities and links into the tables,and a module for reporting the entities that are linked in the tables.

[0128] Although the present invention has been described with referenceto particular embodiments, workers skilled in the art will recognizethat changes may be made in form and detail without departing from thespirit and scope of the invention.

What is claimed is:
 1. A computer executable method of linking an entityin a first application program with an entity in a second applicationprogram, comprising the steps of: receiving at an integration componentinformation contained in a first entity representing the entity in thefirst application program; receiving at the integration componentinformation contained in a second entity representing an entity in thesecond application program; creating a first record in an instance tablefor the first entity, and a second record in the instance table for thesecond entity; and creating a link record in a link table including anentry for the first record and an entry for the second record.
 2. Themethod of claim 1 further comprising: receiving at the integrationcomponent a document including the first entity; posting the document tothe second application program; receiving at the integration componentthe second entity from the second application program.
 3. The method ofclaim 1 wherein creating the first record in the instance table andcreating the second record in the instance table further includescreating an entry in the instance table including a source system ID, anentity ID, and an entity type.
 4. The method of claim 3 wherein creatingthe first and the second records in the instance table further includesassigning a unique ID in the instance table to each record.
 5. Themethod of claim 4 wherein creating a link record in the link tablefurther includes assigning a unique ID for the link record, assigningthe entry for the first record the unique ID of the first record, andassigning the entry for the second record the unique ID of the secondrecord.
 6. The method of claim 3 wherein if the source system ID isprovided as a source system name further performing the steps of:querying the application program that generated the entity; searching asystem table for the system name; identifying a corresponding system IDfor the system name in the system table; and returning to theintegration controller the system ID.
 7. A computer executable methodfor creating a link between linking an entity in a first applicationprogram with an entity in a second application program, comprising thesteps of: gathering a data source connection for each entity to belinked; retrieving a configuration for each of the entities to belinked; for each entity to be linked, performing an ID check todetermine if the ID is valid; determining if a record for the entityexists in an instance table; creating a record in the instance table forthe entity if no record is found; determining if a link record existslinking the two entities in a link table; and creating a link record inthe link table if no corresponding record is found in the link tablelinking the two entities.
 8. The method of claim 7 further comprising:if a record for the entity is found in the instance table, updating therecord in the instance table.
 9. The method of claim 7 wherein creatinga record in the instance table further includes, creating a plurality ofentries in the record including entries for a system ID, an entity ID,and an entity ID.
 10. The method of claim 9 wherein creating a record inthe instance table further includes, assigning the record a unique ID.11. The method of claim 10 wherein creating a record in the link tablefurther includes; assigning the record a unique ID in a first column ofthe link table; inserting the unique ID of the first record in a secondcolumn of the link table; and inserting the unique ID of the secondrecord in a third column of the link table.
 12. The method of claim 9wherein creating a record in the first table includes assigning a timestamp to the record indicative of when the record was created in thetable.
 13. The method of claim 7 wherein performing the ID checkincludes checking a system ID for the entity.
 14. The method of claim 7wherein performing the ID check includes checking an entity ID for theentity.
 15. The method of claim 7 wherein performing the ID checkincludes checking both an entity ID and a system ID for the entity. 16.The method of claim 7 further comprising: prior to the gathering step:receiving a schema containing a plurality of entities to be linked, theplurality of entities being grouped in sets; and validating each set ofentities; and after creating a record in the link table; checking eachset of entities for exceptions; and returning any exceptions to a user.17. The method of claim 16 wherein if an exception is found, writing anentry indicative of the exception to a results file; and whereinreturning any exceptions returns the results file to the user.
 18. Themethod of claim 17 wherein if the set of entities cannot be validatedwriting an error to the results file.
 19. The method of claim 16 whereinchecking for exceptions, checks for business exceptions.
 20. The methodof claim 17 wherein checking for exceptions, checks for systemexceptions.
 21. The method of claim 20 wherein if a system exception isfound returning the results file to the user, and stopping the method.22. A computer executable method for removing a entity link linking anentity in a first application program with an entity in a secondapplication program, comprising the steps of: gathering a data sourceconnection for each entity in the entity link to be removed; retrievinga configuration for each of the entities in the entity link to beremoved; for each entity in the entity link to be removed, performing anID check to determine if the ID is valid; determining if a record forthe entity exists in an instance table; and identifying the record inthe instance table for the entity; identifying any records in a linktable including the records for both entities; and deleting theidentified records from the link table.
 23. The method of claim 22wherein identifying a record in the instance table identifies a uniqueID for each of the entities; and wherein identifying any records linkingthe two entities in the link table identifies those records includingthe unique ID for each entity appearing in the record.
 24. The method ofclaim 23 wherein the gathering the data source connection includesreceiving an indication that a record in the instance table can beremoved or cannot be removed.
 25. The method of claim 24 furtherincluding: searching the link table for other records including theunique ID from the instance table for one of the entities; if no otherrecords are found including the unique ID for one of the entities, andthe received indication indicates that the record can be removed fromthe instance table; then deleting the record associated with the uniqueID for the entity from the instance table.
 26. The method of claim 23further including: searching the link table for other records includingthe unique ID from the instance table for one of the entities; if noother records are found including the unique ID for one of the entities;deleting the record associated with the unique ID for the entity fromthe instance table.
 27. The method of claim 26 further including:creating a copy of the record for the deleted entity in a history table.28. The method of claim 22 wherein the performing the ID check includeschecking a system ID for the entity.
 29. The method of claim 22 whereinthe performing the ID check includes checking an entity ID for theentity.
 30. The method of claim 22 wherein the performing the ID checkincludes checking both an entity ID and a system ID for the entity. 31.A computer executable method for removing a record of an entity from aninstance table in an integration component, comprising the steps of:receiving an indication of the entity to be removed; gathering a datasource connection for the entity to be removed; retrieving aconfiguration for the entity to be removed; performing an ID check todetermine if the ID is valid; identifying the record in the instancetable for entity; and deleting the record from the instance table. 32.The method of claim 31 wherein identifying the record in the instancetable identifies a unique ID for the record.
 33. The method of claim 31further including: identifying in a link table, records including theunique ID from the instance table for the entity; and deleting eachidentified record in the link table.
 34. The method of claim 33 furtherincluding: creating a copy of the record for the deleted entity in ahistory table.
 35. The method of claim 31 wherein performing the IDcheck includes checking a system ID for the entity.
 36. The method ofclaim 31 wherein performing the ID check includes checking an entity IDfor the entity.
 37. The method of claim 31 wherein performing the IDcheck includes checking both an entity ID and a system ID for theentity.
 38. The method of claim 31 further including prior to deletingthe record from the instance table: checking in a link table, recordsincluding the unique ID from the instance table for the entity; and ifno record is found in the link table then deleting the record from theinstance table.
 39. A computer executable method for returning to a usera list of entities that are linked to a first entity comprising thesteps of: receiving the first entity at a linker component; gathering adatasource connection for the entity; receiving a configuration for theentity; verifying that an ID for the entity is valid retrieving a uniqueID for the entity from an instance table; identifying a link table forrecords that contain the unique ID for the entity; and returning to theuser a list of records from the link table that contain the unique IDfor the entity.
 40. The method of claim 39 wherein verifying the IDincludes verifying a system ID for the entity.
 41. The method of claim39 wherein verifying the ID includes verifying an entity ID for theentity.
 42. The method of claim 39 wherein verifying the ID includesverifying both an entity ID and a system ID for the entity.
 43. Themethod of claim 39 wherein returning to the user the list of recordsincludes returning a unique ID of the record in the link table.
 44. Themethod of claim 39 wherein the link table includes two unique ID'srepresenting two entities that are linked together; and whereinreturning to the user the list of records includes returning the uniqueID in the link table that does not represent the received entity. 45.The method of claim 39 further comprising: receiving a second entity atthe linker component, the second entity representing an entity beinglinked to the first entity; executing the gathering, receiving,verifying and retrieving steps for the second entity; whereinidentifying in the link table identifies records that contain both theunique ID for the first entity and the unique ID for the second entity;and wherein returning to the user a list of records returns recordscontaining both unique IDs.
 46. The method of claim 45 wherein returningto the user the list of records includes returning a unique ID for eachrecord in the link table identified.
 47. A computer readable mediumcontaining computer executable instructions that, when executed, cause acomputer to perform the steps of: receiving at an integration componenta document including information contained in a first entityrepresenting an entity in a first application program; receiving at theintegration component information contained in a second entityrepresenting an entity in a second application program; creating a firstrecord in an instance table for the first entity, and a second record inthe instance table for the second entity; and creating a link record ina link table including an entry for the first record and an entry forthe second record.
 48. The computer readable medium of claim 47 whereincreating the first and second records in the instance table furtherincludes creating an entry in the instance table including a sourcesystem ID, an entity ID, and an entity type.
 49. The computer readablemedium of claim 48 wherein creating the first and the second records inthe instance table further includes assigning a unique ID in theinstance table to each record.
 50. The computer readable medium of claim49 wherein creating a link record in the link table further includesassigning a unique ID for the link record, assigning the entry for thefirst record the unique ID of the first record, and assigning the entryfor the second record the unique ID of the second record.
 51. The methodof claim 49 wherein if the source system ID is provided as a sourcesystem name further performing the steps of: querying the applicationprogram that generated the entity; searching a system table for thesystem name; identifying a corresponding system ID for the system namein the system table; and returning to the integration controller thesystem ID.
 52. A computer readable medium containing computer executableinstructions that, when executed, cause a computer to perform the stepsof: receiving an indication of two entities that are to be linked; foreach entity to be linked, performing an ID check to determine if the IDis valid; determining if a record for the entity exists in an instancetable; creating a record in the instance table for the entity if norecord is found; determining if a link record exists linking the twoentities in a link table; and creating a link record in the link tableif no corresponding record is found in the link table linking the twoentities.
 53. The computer readable medium of claim 52 furthercontaining instructions to perform the steps of: gathering a data sourceconnection for each entity to be linked; and retrieving a configurationfor each of the entities to be linked.
 54. The computer readable mediumof claim 52 further containing instructions to perform the steps of: ifa record for the entity is found in the instance table, updating therecord in the instance table.
 55. The computer readable medium of claim52 wherein creating a record in the instance table further includes,creating a plurality of entries in the record including entries for asystem ID, an entity ID, an entity ID and a time stamp indicative ofwhen the record was created in the table.
 56. The computer readablemedium of claim 55 wherein creating a record in the instance tablefurther includes, assigning the record a unique ID.
 57. The computerreadable medium of claim 56 wherein creating a record in the link tablefurther contains instructions to perform the steps of; assigning therecord a unique ID in a first column of the link table; inserting theunique ID of the first record in a second column of the link table; andinserting the unique ID of the second record in a third column of thelink table.
 58. The computer readable medium of claim 55 whereinperforming the ID check includes checking a system ID for the entity.59. The computer readable medium of claim 55 wherein performing the IDcheck includes checking an entity ID for the entity.
 60. The computerreadable medium of claim 52 further containing instructions to performthe steps of: wherein receiving an indication receives a schemacontaining a plurality of entities to be linked, the plurality ofentities being grouped in sets; validating each set of entities; andafter creating a record in the link table; checking each set of entitiesfor exceptions; writing any exceptions to a results file; and returningthe results file to a user.
 61. The computer readable medium of claim 60wherein if the set of entities cannot be validated writing an error tothe results file.
 62. The computer readable medium of claim 60 whereinchecking for exceptions, checks for business exceptions.
 63. Thecomputer readable medium of claim 60 wherein checking for exceptions,checks for system exceptions.
 64. The computer readable medium of claim63 wherein if a system exception is found returning the results file tothe user, and stopping the method.
 65. A computer readable mediumcontaining computer executable instructions that, when executed, cause acomputer to perform the steps of: receiving an indication of twoentities whose link is to be removed; for each entity in the entity linkto be removed; performing an ID check to determine if the ID is valid;determining if a record for the entity exists in an instance table; andidentifying the record in the instance table for the entity; identifyingany records in a link table including the records for both entities; anddeleting the identified records from the link table.
 66. The computerreadable medium of claim 65 further containing instructions to performthe steps of: gathering a data source connection for each entity in theentity link to be removed; and retrieving a configuration for each ofthe entities in the entity link to be removed.
 67. The computer readablemedium of claim 65 wherein identifying a record in the instance tableidentifies a unique ID for each of the entities; and wherein identifyingany records linking the two entities in the link table identifies thoserecords including the unique ID for each entity appearing in the record.68. The computer readable medium of claim 67 further containinginstructions to perform the steps of: searching the link table for otherrecords including the unique ID from the instance table for one of theentities; if no other records are found including the unique ID for oneof the entities; deleting the record associated with the unique ID forthe entity from the instance table.
 69. The computer readable medium ofclaim 68 further including: creating a copy of the record for thedeleted entity in a history table.
 70. The computer readable medium ofclaim 65 wherein the performing the ID check includes checking both anentity ID and a system ID for the entity.
 71. A computer readable mediumcontaining computer executable instructions that, when executed, cause acomputer to perform the steps of: receiving an indication of an entityto be removed; gathering a data source connection for the entity to beremoved; retrieving a configuration for the entity to be removed;performing an ID check to determine if the ID is valid; identifying therecord in the instance table for entity; and deleting the record fromthe instance table.
 72. The computer readable medium of claim 71 whereinidentifying the record in the instance table identifies a unique ID forthe record.
 73. The computer readable medium of claim 71 furthercontaining instructions to perform the steps of: identifying in a linktable, records including the unique ID from the instance table for theentity; and deleting each identified record in the link table.
 74. Thecomputer readable medium of claim 73 further containing instructions toperform the steps of: creating a copy of the record for the deletedentity in a history table.
 75. A computer readable medium containingcomputer executable instructions that, when executed, cause a computerto perform the steps of: receiving a first entity at a linker component;gathering a datasource connection for the entity; receiving aconfiguration for the entity; verifying that an ID for the entity isvalid retrieving a unique ID for the entity from an instance table;identifying a link table for records that contain the unique ID for theentity; and returning to the user a list of records from the link tablethat contain the unique ID for the entity.
 76. The computer readablemedium of claim 75 wherein returning to the user the list of recordsincludes returning a unique ID of the record in the link table.
 73. Thecomputer readable medium of claim 71 wherein the link table includes twounique ID's representing two entities that are linked together; andwherein returning to the user the list of records includes returning theunique ID in the link table that does not represent the received entity.78. The computer readable medium of claim 75 further containinginstructions to perform the steps of: receiving a second entity at thelinker component, the second entity representing an entity being linkedto the first entity; executing the gathering, receiving, verifying andretrieving steps for the second entity; wherein identifying in the linktable identifies records that contain both the unique ID for the firstentity and the unique ID for the second entity; and wherein returning tothe user a list of records returns records containing both unique IDs.79. The computer readable medium of claim 78 wherein returning to theuser the list of records includes returning a unique ID for each recordin the link table identified.